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(54) MOVEMENT CONTROL SYSTEM FOR INSTRUCTING POSITION OF ROBOT 

(57)Abstract: 

PROBLEM TO BE SOLVED: To make instructing 
operation, which makes good use of a visual sensor and a 
visual target, efficient. 

SOLUTION: A desired instruction position arrival state 
representation data input means la which makes good 
use of the visual sensor previously inputs data, 
representing the relative position relation between the 
visual target Bl and a robot arm tip 2a in a state 2b' 
. wherein the TCP 2b of the robot 2 has finished moving 
to a desired instruction position Al, etc., to a memory 4. 
An operator operates a robot manual operation means 5 
such as a jog feed button to make a shaft movement 
control means 6 control respective shafts, thereby 
starting jog movement. Further, the mode is shifted from 
the job movement mode to an autonomous movement 
mode and the TCP 2b is autonomously moved to the 
desired instruction position Al, etc. Control over the 
autonomous movement is performed on the basis of the 
desired instruction position arrival state representation 
data and data obtained by a robot arm tip-visual target relative position recognizing means lb 
which utilizes the visual sensor. A target means is actualized by a mark coordinate system or 




light spot projection means. 



CLAIMS 



[Claim(s)] 

[Claim 1]A visual sensor provided with an image processing means which processes a 
picture acquired through a camera means attached to a hand part of a robot and this robot, 
and this camera means, In a movement control system of a robot for moving said robot to 
a position wishing instruction using a control means of a system containing a robot 
control device, A position attainment state storage step wishing instruction which makes 
said control means memorize position attainment state representation data wishing 
instruction which expresses a position attainment state wishing instruction about at least 
one position wishing instruction, An impaction efficiency stage wishing instruction of 
moving said robot to said position wishing instruction by movement controls of said 
robot by said control means, Instruction positional data including a teaching-positions 
storage step which said control means is made to memorize said impaction efficiency 
stage wishing instruction, Include an autonomous movement execution phase to which 
movement in this position wishing instruction is made to carry out autonomously based 
on said position attainment state representation data wishing instruction, and said position 
attainment state representation data wishing instruction, It is what is acquired through 
seeing from said visual sensor expressed using a visual target means which can be 
recognized with said visual sensor, and said visual sensor recognizing a position 
attainment equivalent state wishing instruction equivalent to a position attainment state 
wishing instruction, In said autonomous movement stage said visual target means, It is 
what provides a guidance visual target means for showing said robot to said position 
wishing instruction through being recognized by said visual sensor, By software 
processing within said control means, movement controls of said robot in said 
autonomous movement stage are performed, and said software processing, Said method 
with which a recognition state of said guidance visual target means by said visual sensor 
includes processing for deriving said robot so that it may be in agreement with a 
recognition state corresponding to said position attainment state representation data 
wishing instruction. 

[Claim 2] Said visual target means is a mark means including a mark coordinate system 
which can be recognized with said visual sensor, Position attainment state representation 
data wishing instruction memorized by said position attainment state storage step wishing 
instruction, Data showing a position and a posture of said mark coordinate system on a 
picture is included, and said mark means, A movement control system for position 
instruction of a robot indicated to claim 1 which is what prepares said mark coordinate 
system for said position wishing instruction, and a position with fixed relative relation in 
order to provide said guidance visual target means. 

[Claim 3]A movement control system for position instruction of a robot indicated to 
claim 2 with which it is a mark member which said mark means can fix on a field of a 
representation work made into an object of teaching work, and said mark coordinate 
system is drawn on this mark member. 

[Claim 4]In [ said visual target means is the light spot formed on a floodlighting side of 
an optical beam light projection means supported by said robot, and ] said position 
attainment state storage step wishing instruction, In order to prepare said position 



attainment equivalent state wishing instruction to said visual sensor, A reference 
floodlighting side for forming said light spot in a position which is in agreement at an 
instruction target point of said robot is arranged, and it lets it pass to recognize light spot 
formed on this reference floodlighting side with said visual sensor, It is acquired by said 
position attainment state representation data wishing instruction so that data showing a 
position on a picture of said light spot may be included, and said autonomous movement 
stage, It is what is performed where said optical beam floodlighting direction which 
removed said reference floodlighting side and was seen from said camera in said position 
attainment equivalent state wishing instruction is maintained, Movement controls of said 
robot in said autonomous movement execution phase, It is carried out, using light spot 
which floodlights said optical beam and is formed on a field where said position wishing 
instruction exists as said guidance visual target means, and a recognition state of said 
light spot by said visual sensor said software processing for movement controls of said 
robot, A movement control system for position instruction of a robot indicated to claim 1 
include processing for deriving said robot so that it may be in agreement with a 
recognition state corresponding to said position attainment state representation data 
wishing instruction. 

[Claim 5] A stage of jogging before an autonomous movement execution phase to said 
position wishing autonomous instruction is included in a move execution phase to said 
position wishing instruction, A movement control system for position instruction of a 
robot indicated in any 1 paragraph of claim 1 - claim 4 to which shift to a move execution 
phase from said jogging stage to said position wishing autonomous instruction is 
automatically performed by said control means based on an output of said visual sensor. 
[Claim 6]Data showing a position on a picture of said guidance visual target means by 
which software processing in an autonomous movement execution phase to said position 
wishing instruction is recognized by said visual sensor at the time concerned, Processing 
which compares said position attainment state representation data wishing instruction, 
and processing for carrying out the movement controls of each axis of a robot based on 
said comparison result, It is what repeats sequentially processing which judges 
completion / un-completing of a position attainment state wishing instruction until a 
judgment of position attainment state completion wishing instruction is made, A 
movement control system for position instruction of a robot with which said teaching- 
positions storage step is performed after a judgment of this position attainment state 
completion wishing instruction is made and which was indicated in any 1 paragraph of 
claim 1 - claim 5. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] Concerning the movement control system for position instruction 
of a robot in more detail, this invention relates to the movement control system for 
moving a robot to the position wishing instruction quickly using a visual target and the 
visual sensor which recognizes this. 
[0002] 



[Description of the Prior Art]Most generally as a position teaching method of a robot, the 
method by jogging (robot motion by a manual input) is used. According to this teaching 
method, an operator operates the jogging button of a teaching control panel, and moves a 
robot to the position wishing instruction, and the work of teaching the position at that 
time to a robot is done. When off-line instruction is made and coarse position instruction 
has ended, or when making correction of teaching positions, fine adjustment of the 
teaching positions by jogging is performed. 

[0003]use any - since the work for which a robot is moved to it locating [ to wish ] is 
done by viewing with slight accuracy in relative position and posture relation of a robot 
hand (or end effector) and the point wishing [ instruction ], an operator takes skill to it. 
As for the number of points (a position and a posture) to be taught, since it is usually to 
attain to a large number, it is not new to spend great time on teaching work, either. The 
teaching work depending on viewing of an operator tended to produce variation for 
instruction accuracy, and there was a problem also in respect of reliability. 
[0004] 

[Problem(s) to be Solved by the Invention]This invention is made in order to conquer 
such a situation. That is, the purpose of this invention eases the burden which takes for an 
operator at the time of teaching work, and there is in providing the movement control 
system for position instruction of the robot which enabled improvement in the efficiency 
of teaching work, and reliability. 
[0005] 

[Means for Solving the Problem]This invention was made in view of a problem of the 
above-mentioned conventional technology, and an aforementioned problem is solved by 
enabling it to move a robot to a position wishing instruction autonomously using a visual 
sensor and a suitable visual target means. A position attainment state storage step wishing 
instruction which a control means of a system which contains a robot and a visual sensor 
in many stages of a method of this invention for position attainment state representation 
data wishing instruction expressing a position attainment state wishing instruction of a 
robot is made to memorize, An impaction efficiency stage wishing instruction of moving 
a robot to a position wishing instruction by movement controls of a robot by this control 
means, and a teaching-positions storage step which makes a control means memorize 
instruction positional data are included. 

[0006]And it is the important feature of this invention that an autonomous movement 
execution phase made to carry out to this impaction efficiency stage wishing instruction 
autonomously based on position attainment state representation data wishing instruction 
which had movement in a position wishing instruction memorized is included. 
[0007]In order to make it possible, this autonomous movement position attainment state 
representation data wishing instruction, It is acquired through seeing from a visual sensor 
expressed using a visual target means which can be recognized with a visual sensor, and 
said visual sensor recognizing a position attainment equivalent state wishing instruction 
equivalent to a position attainment state wishing instruction. 

[0008]A visual target means is used in an autonomous movement stage as a guidance 
visual target means for showing a robot to said position wishing instruction through being 
recognized by a visual sensor. 

[0009] Software processing performed within a control means for autonomous movement 
is performed so that a recognition state of a guidance visual target means by a visual 



sensor may be in agreement with a recognition state corresponding to position attainment 
state representation data wishing instruction and a robot may be derived. The visual target 
means can take a gestalt of a mark means including a mark coordinate system which can 
be recognized with a visual sensor. In that case, data showing a position and a posture of 
a mark coordinate system on a picture will be contained in position attainment state 
representation data wishing instruction memorized by a position attainment state storage 
step wishing instruction. On the other hand, this mark means is used in order to prepare a 
mark coordinate system for a position with fixed in order to provide a guidance visual 
target means for autonomous movement of a robot relative relation as a position wishing 
instruction. And a robot is derived so that a recognition state of a mark coordinate system 
by a visual sensor may be in agreement with a recognition state corresponding to position 
attainment state representation data wishing instruction by movement controls based on 
software processing performed within a control means. 

[001 0]A mark member fixable on a field of a representation work made into an object of 
teaching work can be used for a mark means. On this mark member, a mark coordinate 
system is drawn by a dot pattern etc. 

[001 l]There is light spot formed on a floodlighting side of an optical beam light 
projection means supported by robot as a gestalt of another available visual target means. 
When using such light spot as a visual target means, in order to prepare a position 
attainment equivalent state wishing instruction to a visual sensor in a position attainment 
state storage step wishing instruction, it is an instruction target point (a point which it is 
going to coincide with a position wishing instruction) of a robot about light spot. Usually, 
it is tool point;TCP. A reference floodlighting side for forming in a position in agreement 
is arranged. 

[0012]The optical beam floodlighting direction is adjusted and light spot is formed in an 
instruction target point position on this reference floodlighting side. This state is 
equivalent to a teaching-positions attainment equivalent state for a visual sensor. Then, 
light spot is recognized by a visual sensor, and position attainment state representation 
data wishing instruction is acquired so that data showing a position on a picture of light 
spot may be included. 

[00 13] An autonomous movement stage is performed where the optical beam 
floodlighting direction which removed a reference floodlighting side and was seen from a 
camera in a position attainment equivalent state wishing instruction is maintained. That 
is, movement controls of a robot in an autonomous movement execution phase are 
performed, using light spot which floodlights an optical beam and is formed on a field 
where a position wishing instruction exists as a guidance visual target means. Software 
processing for movement controls of a robot is performed so that a recognition state of 
light spot by a visual sensor may be in agreement with a recognition state corresponding 
to position attainment state representation data wishing instruction and a robot may be 
derived. 

[0014]In advance of autonomous movement to a position wishing instruction, it may be 
made to jog in order to perform preliminary approach to a position wishing instruction. 
As for shift to an autonomous movement stage from a jogging stage, in that case, it is 
preferred to be automatically carried out by the processing within a control means based 
on an output of a visual sensor. 

[0015]Software processing in an autonomous movement execution phase to a position 



wishing instruction, Processing which compares with position attainment state 
representation data wishing instruction data showing a position on a picture of a guidance 
visual target means recognized by a visual sensor at the time concerned, Suppose that 
processing for carrying out the movement controls of each axis of a robot based on a 
comparison result and processing which judges completion / un-completing of a position 
attainment state wishing instruction are repeated sequentially until a judgment of position 
attainment state completion wishing instruction is made. 

[0016]Data of a position on a picture of a visual target means observed in this invention 
when a position wishing instruction is reached is inputted beforehand, and at the time of 
autonomous movement to a position wishing instruction. A visual target means is used 
like a navigation index of robot movement by observing with a visual sensor, and a robot 
is made to turn and derive to a position wishing instruction. 
[0017]By this feature, a position attainment state wishing instruction is attained 
efficiently, without applying a burden to an operator. The highly convenient nature of a 
robot at the time of position teaching work can also be further raised by performing shift 
to autonomous movement from jogging with the automatic change in a system. 
Therefore, even when a large number [ the number of positions wishing instruction ], 
there are few burdens of teaching work and they end. 
[0018] 

[Embodiment of the Invention]DrawingJ_is a figure which illustrates notionally the 
overview of the movement control system for the position instruction of a robot 
according to this invention. The numerals la express the position attainment state 
representation data input means wishing instruction which inputs the data expressing the 
position attainment state wishing instruction. This position attainment state representation 
data input means la wishing instruction, Visual target Bl Data expressing the relative 
physical relationship of the visual target Bl in the state where movement in the position 
wishing instruction of the robot [ / else ] 2 was completed, and the robot hand 2a. (it is 
hereafter called "the position attainment state representation data wishing instruction".) -- 
it has the function to input into the memory 4 beforehand. 

[0019]The position wishing instruction is illustrated by the four numerals Al on the field 
3a of the representation work 3 - A4. Bl drawn Al - near each [ of A4 ] - B4 are given by 
the position Al wishing instruction - A4 representing the position which Al - A4 express 
which are visual target means and each correspond, and a fixed relation. The example of 
a visual target means is mentioned later. In the usual gestalt, each visual target position 
Bl - B4 and each position Al wishing instruction - A4 are given so that coincidence may 
not be carried out, although it is close, but they can also coincide both depending on the 
case. 

[0020]The numerals 2a and 2b express the end point position and instruction target point 
(usually tool point; TCP) of the robot respectively. In the following explanation, the end 
point position 2a of a robot shall be represented with the starting point of the coordinate 
system set up on the flange at the tip of a final arm. 

[0021] An operator operates the robot manual operation means (robot motion instruction 
manual input means; teaching control panel etc.) 5, such as a jogging button, makes the 
shaft movement control means 6 control each axis of the robot 2, and makes movement in 
the position wishing instruction start in the time of actual work. The movement controls 
for autonomous movement in the position wishing instruction from the time of this move 



start or a certain point in time after a move start are performed. The position attainment 
state representation data wishing instruction into which the movement controls for this 
autonomous movement were inputted by the position attainment state representation data 
input means 1 a wishing instruction, It is carried out based on the relative-position- 
recognition data showing the relative physical relationship between the robot hand and 
visual target means (camera fixed to this) at each time. 

[0022]The latter data is given once [ at least ] in the process of movement controls by the 
relative-position-recognition means lb between robot hand visual targets, in the position 
attainment state wishing instruction — naturally - TCP2b - the position wishing 
instruction - one is in agreement with Al (or A2 - A4 one) (it illustrates by numerals 
2b'). 

[0023]The position attainment state representation data input means la wishing 
instruction and the relative-position-recognition means lb are materialized in the form 
where a visual sensor is used so that it may mention later. The visual target means Bl - 
B4 are materialized in the form of the light spot formed of the mark coordinate system or 
laser beam which can be recognized with a visual sensor. In the following explanation, 
the gestalt which materializes a visual target means in the form of a mark coordinate 
system is called "the method 1 ", and the gestalt materialized in the form of the light spot 
formed of a laser beam is called "the method 2." 

[0024] drawing 2 showed the outline of the hardware constitutions of the system used by 
this embodiment with the important section block diagram centering on a robot control 
device ~ it is. The robot control device to which the whole was directed with the 
numerals 30 was equipped with the processor board 31, and this processor board 31 is 
provided with the central processing unit (henceforth CPU) 31a, ROM31b, and RAM31c 
which consist of microprocessors. 

[0025]CPU31a controls the whole robot control device according to the system program 
stored in ROM3 lb. A program, a related preset value, etc. which defined the processing 
by the side of a robot required to perform autonomous movement to the position wishing 
instruction else [, such as a created operation program and various preset values, ] 
according to the method 1 or the method 2 are stored in RAM31c. A part of RAM31c is 
used for temporary data storage for the computation etc. which CPU3 la performs. The 
hard disk drive etc. which were suitably prepared as an external device are used for 
preservation of program data or a preset value. 

[0026]The processor board 3 1 is combined with the bus 37, and instructions and transfer 
of data are performed among other portions in the robot control device 30 via this bus 
combination. First, the digital-servo control circuit 32 is connected to the processor board 
31, and the servo motors 51-56 are driven via the servo amplifier 33 in response to the 
instructions from CPU31a. The servo motors 51-56 which operate each axis are built in 
the mechanism part of each axis of the robot 2. 

[0027]The serial port 34 having a communication interface is connected to the teaching 
control panel 40, the image processing device 20, and the laser generator 60 with a liquid 
crystal display section while being combined with the bus 37. However, the laser 
generator 60 is used by the method 2, and is unnecessary in the method 1. 
[0028]The teaching control panel 40 has the size and weight which are the grades whose 
carrying is possible with an operator, and the jogging button etc. which are used as a 
robot manual operation means are provided on the panel. In addition, the input/output 



device 35 for digital signals (digital I/O) and the input/output device 36 for analog signals 
(analog I/O) are combined with the bus 37. When signal transfer with an end effector is 
required, the control section of an end effector is connected to digital 1/035 or analog 
1/036. In the example mentioned later, since the arc welding robot's application is 
considered, the electric power unit of an arc welding torch is connected to digital 1/035. 
[0029]The image processing device 20 is the usual thing which made CPU carry out bus 
combination of program memory, a frame memory, an image processing processor, data 
memory, the camera interface, etc. The camera 21 is connected to the image processing 
device 20 via the camera interface. This camera is used for the photography for acquiring 
the picture of a visual target means in the mode mentioned later. The program data for 
image analyses needed by the method 1 or the method 2 is stored in program memory. 
[0030] Drawing 3 is a figure showing the outline composition of the panel surface of the 
teaching control panel 40. The display screen 41 is a liquid crystal display, and the 
detailed data of a movement command program, etc. change it, and it is displayed. The 
function key 42 is a key which selects the menu displayed on the lower end part of the 
display screen 41. The teaching control panel effective switch 43 is a switch which 
switches whether operation of the teaching control panel 40 is effective or invalid. 
[003 l]The emergency stop button 44 is a button to which the emergency stop of the 
operation of the robot 2 is carried out. The cursor key 45 is a key which moves the cursor 
displayed on the display screen 41 . A numerical keypad and other keys are provided in 
the ten key part 46, and input of a numerical value and a character, deletion, etc. can be 
performed to it. 

[0032] Although a series of jogging buttons 47 (J1-J6) are buttons which specify 
advancing side by side / hand of cut, and the direction of +-, and input a movement 
command in the normal mode which jogs a conventional system, According to this 
embodiment (autonomous movement mode), it is used as an autonomous movement 
command input means to the position wishing instruction so that it may mention later. 
The details of an embodiment are explained below about the method 1 and the method 2 
on the assumption that the above matter. 
[0033][Method 1] 

[1] The schematic diagram 4 is a figure explaining operation of the autonomous 
movement in the embodiment by the method 1 . On the field 3 a of the representation work 
directed with the numerals 3 as well as drawing 1 , a number (here four pieces) 
corresponding to the number of the teaching points Al - A4 to wish of mark members 
MK1-MK4 are stuck. On each mark member, the same mark coordinate system is drawn 
by the dot pattern so that it may mention later. Mark member MK1 is stuck by the 
position and the posture in which it corresponds to the position and posture of the point 
Al wishing [ instruction ] correctly. Similarly, the mark members MK2-MK4 are stuck 
by the position and the posture in which it corresponds to the point A2 wishing [ 
instruction ] - the position and posture of A4 correctly respectively. 
[0034]The robot which carried out representation only of the hand part circumference 
carries the welding torch 2c and the camera 21 in the hand part, and TCP2b is set up at 
the tip of welding torch 2c. By this embodiment, a robot is left from the move starting 
position Ps, autonomous movement of the TCP2b set up at the tip of the welding torch 2c 
is carried out to the point Al wishing [ instruction ] - A4 one by one, and the case where 
position instruction is performed in each position wishing instruction is considered. 



Generally the starting position Ps of autonomous movement is arbitrary, if mark MK1 of 
the beginning is a position included in the view of the camera 21 . 
[0035]Drawing_5_is a figure explaining the composition of the mark member used by the 
embodiment by the method 1 . Reference of the figure constitutes mark coordinate system 
sigmaM from the five circular dots DO arranged in the shape of a lattice by the interval a 
on the mark member MK, Dl, D-l, D2, and D-2. Dot space a is positive constant value. 
Therefore, the center position of each dot is set to DO (0, 0, 0), Dl (a, 0, 0), D-l (- a, 0, 
0), D2 (a, a, 0), and D-2 (- a, a, 0). As long as it can express a three-dimensional 
rectangular coordinate system, a mark coordinate system may consist of other patterns. 
[0036]Hole MH is provided in the regular position where the mark member MK is 
suitable. This hole MH points to the position wishing instruction, and when sticking on 
the field 3 a of the representation work 3, a sticking position is chosen so that that 
representative point (for example, center) may be in agreement with the position wishing 
instruction. A pasting posture is chosen with reference to direction of mark coordinate 
system sigmaM. That is, if pasting postures differ, the postures taught later also differ. 
For example, since the pasting posture of MK1 and MK4 in drawing 4 is different 90 
degrees, it becomes what differed also in the posture taught later 90 degrees. 
[003 7] [2] Preparation (the calibration of a camera, and acquisition of the position 
attainment state representation data wishing instruction) 

1 . Perform the calibration of the camera with the application of suitable calibration 
method in advance of working starting. Although various techniques are known by the 
calibration of the camera, the mark member shown here can also be used. The outline is 
described on the convenience of explanation, and in the back. 

[0038]2. Perform acquisition of the position attainment state representation data wishing 
instruction. First, the mark member MK is fixed to a suitable position. Of course, one of 
the mark members stuck on the representation work 3 can also be used as it is. First, by 
jog operation of the normal mode (conventional system), a robot is moved, tool point 2b 
is coincided with representative point MA of hole MH of the mark member MK, and it is 
made to take the posture wishing instruction. It means that the position attainment state 
wishing instruction was realized about the mark member MK by this. 
[003 9] 3. If the position attainment state wishing instruction is realized about the mark 
member MK, photography with the camera 21 will be made to perform via the robot 
control device 30, and the picture for creating the position attainment state representation 
data wishing instruction will be acquired. 

4. Image processing of the acquired picture is carried out within the image processing 
device 20, and the data showing the relative positional attitude of mark coordinate system 
sigmaM to the camera coordinate system of the camera 21 is created (it mentions later for 
details). 

[0040] [3] Start movement in the position wishing autonomous instruction from the 
approach starting position Ps (it expresses in a flange position) shown in processing 
drawing 4 at the time of the position autonomous movement wishing instruction in the 
method 1, and explain the algorithm of the processing for attaining movement in the 
position Pt (it expresses in a flange position) wishing instruction. In the following 
explanation, since a vector is expressed, sign <> is used. 

1 . Skeleton drawing 7 of an algorithm describes the skeleton of an algorithm required for 
movement in the autonomous position wishing instruction. The current position (position 



at each [ under movement in the position wishing instruction ] time) TO of a flange, and 
the present relative position (relative position at each [ in the moving process to the 
position wishing instruction ] time) MO of a camera coordinate system and a mark 
coordinate system, As [ describe / the geometric relation with the flange position Tg and 
relative-position (target relative position) Mg of a camera coordinate system and a mark 
coordinate system in the position attainment state wishing instruction in the position 
attainment state wishing instruction / into drawing 4 ] 

[0041] And Mg is equivalent to the position attainment state representation data wishing 
instruction acquired by the above-mentioned preparatory work. C is taught by a camera 
calibration. The fundamental equation which specifies the relation between these [ TO ], 
MO, Tg, and Mg is as follows. 

TO CMO =Tg CMg ... (1), therefore the following formula (2) which solved this about Tg 
turn into a basic equation showing the moving-target position on the rectangular 
coordinate system of a robot. 
Tg =T0 CMO Mg'C 1 ... (2) 

If the position instruction which makes this Tg a final moving-target point is given to a 
servo, a robot can be turned to the position wishing instruction and can be moved 
autonomously. Therefore, defining the algorithm of the autonomous movement to the 
position wishing instruction results in the problem which asks for the right-hand side of 
(2) types concretely. 

[0042](2) Among the right-hand side of a formula, TO expresses the present position data 
of the robot, and is data of the character obtained within a robot control device at any 
time. The data of C is separately gained by a suitable camera calibration (the example of 
a calibration is mentioned later). Then, it explains from how to calculate MO and Mg. 
[0043]2. Each of equations MO of MO and Mg and Mg expresses the position and the 
posture of mark coordinate system sigmaM seen from the camera coordinate system (at 
the time of the completion of the present and approach). 

[0044] Since it is positive constant value, the arrangement interval a of the circular dots 
DO which constitute mark coordinate system sigmaM (refer to drawing 5 ) used by this 
embodiment, Dl, D-l, D2, and D-2 is a> 0... It is (3). M0 and Mg are the homogeneous 
transformation processions of 4x4 expressing the relation between the position and 
posture between three-dimensional rectangular coordinate systems, and can be placed 
like a following formula (4). 
[0045] 
[Equation 1] 



(4) 



Here, 3x3 processions for which RM expresses rotation, and 1M are 3x1 processions 
(vector) showing a position. It considers asking for the equation which RM and 1M fill 
hereafter as an equation which M0 and Mg fill. First, the following formula (5) and (6) 
defines a vector <el> and <e2>. 
[0046] 
[Equation 2] 



As for the information acquired from a visual sensor about mark coordinate system 
sigmaM, the center of each circular dots DO, Dl, D-l, D2, and D-2 aims (sight line 
direction) to be in sight from the starting point of a camera coordinate system. These 
directions can be expressed with five unit vectors <d0>, <dl>, <d-l>, <d2>, and <d-2> 
as shown in drawing 6 . 

[0047]deltaij defined by the following formula (7) is introduced about the inner product 
of these vectors. 

deltaij=<di>- <dj> (i, j=-2, -1, 0, 1, 2) ... (7) - here, since all the length of <d0>, <dl>, 

<d-l>, <d2>, and <d-2> is 1, the absolute value of deltaij does not exceed 1 . 

| deltaij |<=1 (a necessary and sufficient condition for equal mark formation is i=j) ... (8) 

Although distance from the camera coordinate system starting point to each dot center is 

unknown, this is placed with ti (i=-2, -1, 0, 1, 2). Naturally these take a positive value. 

[0048] 

ti >0 (i=-2, -1, 0, 1, 2) ... If (9) is carried out, following equation (10) - (12) will be 
obtained. 

tO <d0> =1M ... (10) tl <dl> =lM+aRM <el> ... (11a) 
t-Kd-l> =1M -aRM <el> ... (1 lb) 
t2 <d2> =tl <dl>+aRM <e2> ... (12a) 
t-2<d-2> =t-Kd-l>+aRM <e2> ... (12b) 

Therefore, if an equation of these is solved about RM and 1M and it goes, calculating ti, 
M0 and Mg can be defined. A concrete solution will be explained by a paragraph of how 
to calculate 4.M0 and Mg, and C showing a position and a posture of a camera coordinate 
system is described briefly. 

[0049]3. The equation C of C is a procession expressing a position and a posture of a 
camera coordinate system seen from a flange, and the data is obtained by calibration of 
the camera 21. Although various techniques are publicly known about a calibration and 
they can be used, even if it uses mark coordinate system sigmaM, a calibration of the 
camera 21 can also be performed. Here, only the main point is described briefly. 
[0050]In three different points as shown with the numerals CB1-CB3 in drawing 4 , 
photography with the camera 21 is performed respectively, image processing is 
performed and the data Pi (i= 1, 2, 3) about a relative position between the starting points 
of mark coordinate system sigmaM and a camera coordinate system is obtained. If data 
Ti (i= 1, 2, 3) of a robot position at the time of they and each photography is combined, 
the following 3 sets of equations will be obtained. The point which calculates Pi (i= 1, 2, 
3) is the same as that of M0 or Mg. 
Tl CP1 =T2 CP2 =T3 CP3 ... (13) 

Under conditions from which three postures of a robot of CB1-CB3 differ mutually, it is 

possible to solve this equation about C. This condition is set to (14). 

Conditions: Tl and T2 differ from each posture ingredient of T3 mutually. ... Data of C 



(14) Called for is memorized by a robot control device or image processing device. 
About a solution of an equation, it explains supplementarily depending on how to ask for 
5.C. 

[005 1]4. In MO and Mg asking, and solving equation (10) - (12) here and there, and 
calculating RM and 1M, consider calculating ti (i=-2, -1, 0, 1, 2) first. Therefore, a 
following formula (15) is replaced. 
[0052] 
[Equation 3] 

< rM)>= rK = RM~' < IH> • • • (1 5) 

L rMz J 



And equation (10) - (1 1) is transformed like following formula (16) - (18) using this. 
<rM> =t0 RM" 1 <d0> ... <rM>+a<el >=tl RM" 1 <dl> ... <rM>-a<el >=t-lRM"' <d-l> ... 
(18) ~ here, (17) (16) If RM" 1 is cautious of it being a procession showing rotation, a 
following formula (19) will be materialized. 

(RM" 1 <di>) -(RM" 1 <dj>) =<di> - <dj> (i, j=-2, -1, 0, 1, 2) ... If this (19) (19) type, and 
(5) - (8) type and (16) - (18) type are used, following expression-of-relations (20) - (23) 
will be obtained. 
[0053] 

||<rM> || 2 =t0 2 ... (20) ||<rM> || 2 +2 arMx+a 2 =tl 2 ... (21a) 
||||[ <rM> ] 2 -2 arMx+a 2 =t-l 2 ... (21b) 
||<rM> || 2 +arMx=t OtldeltaO and 1 — (22a) 
||<rM> || 2 -arMx=tO t-ldeltaO and -1 — (22b) 
delta 0, 1 2 <1, delta 0, and -1 2 <1 — (23) 

If it arranges carefully from these formulas for ||<rM> || 2 +arMx, adeltaO, 1 and ||<rM> || 2 - 
arMx, and adeltaO and -1 to become a same sign from conditions (3), (9), a formula (22a), 
and (22b) respectively, following formula (24a) - (25b) will be obtained. 
[0054] 
[Equation 4] 

II < r M > || 2 + a rMx=a a^rMy 2 + rMz 2 ♦■•C2 4a) 



|| < r M > || 2 - a rMx= a a- ^rMy 2 + rMz 2 



/J 1 -s 



sJ 1-S0.-1 2 

If a formula (24a) and (24b) change further, the following formula (26) and (27) will be 
obtained. 

||<rM> || 2 =(a 2 beta+ 2 )/(l+beta- 2 ) ... (26) rMx=(abeta+beta-)/(l+beta- 2 ) ... (27) however 
beta+, and beta- shall be defined by a following formula (28a) and (28b). 
beta+ = (alpha++alpha-) 12... (28a) 



beta- = (alpha+-alpha-) / 2 ... (28b) 

And if the formula (26) and (27) is substituted for the formula (20), (21a), and (21 b) and 
the conditions (3) and (9) are used, tO, tl, and t-1 can be found with the following 
formula (29), (30a), and (30b). 
[0055] 

[Equation 5] 



1 + ?-' 



t2, t-2, <1M>, and RM are further calculated as follows using these. 

[0056] 

[Equation 6] 

1 1 — t -1 s 1 . -1 

t2 m t1 ***(30c1) 

t1 ei.2- t-1 C-t.2 
(tl 81,2- t-i 8-i,2»»0a)»fr) 



t-1 5 -1, -2- 1 161.-2 

(tl 81,2- t-1 6-1,2= 0CMM) 



< I H>= t0< d0> 

<RM>= «f1 Xf2 Xf3 » 



• • (3 Od) 

• • (30e) 



iu <f 1 >== - 



(tKd1>-t-Kd-1» 



- (t2<d2>-tKd1» 
Ctl 81.2- t-1 8-1, 2*0©**) 



- C t -2< d-Z>- t-t< d-1 » 
L (tl 81,2- t-1 8-1.2=0©^) 



< f 3 > = < f 1>x< f 2> 



5. Solve the above-mentioned equation (13) under the conditions (14) of the how [ to ask 
for C ] above-mentioned. Modification of an equation (13) will obtain the following (31) 
and (32). 

C'Tl -'T3 C=P1 P3 - 1 ... C ! T2 _1 T3 C=P2 P3 _1 ... (32) - replacement which followed 



following formula (33) - (37) here is performed. (31) 
[0057] 
[Equation 7] 

c I def 

(3 3) 



R3 
*0 



Then, the equation (31) and (32) is decomposed into following equation (38) - (41). 
RC _1 R1 RC =R2 ... (38) RC'R3 RC =R4 ... (39) <(I-R1) 1C> =<11>-RC <12 > ... (40) <(I- 
R3) 1C> =<13>-RC <14> ... (41) 

Therefore, it becomes a target to calculate RC and <1C>. Here, a form of an equation (38) 

and (39) shows that both similar transformation processions expressing the similarity Rl 

during a rotation procession - R2, R3 - R4, and those similarity relations are RC. Then, 

the following formula (42) and (43) is obtained considering a vector showing a hand of 

cut of Ri (i= 1, 2, 3, 4) as <vi> (i= 1, 2, 3, 4). Here, <vi> (i= 1, 2, 3, 4) can be uniquely 

defined from conditions (14), a definitional equation (34), and (36). 

<vl> =RC <v2 > ... (42) <v3> =RC <v4> ... (43) 

RC can be calculated if a relation of a following formula (44) is used. 

(<vl>, <v3>, and <vl> x <v3>) =RC (<v2>, <v4>, and <v2> x <v4>) ... About (44) 

equations (40) and (41), a form of a following formula (45) is used at a bundle, and if a 

least-squares method is applied, <1C> will be calculated. 

[0058] 

[Equation 8] 



P - R, i<,c>. r <M> i -ro r <,2> i 

I_! ~ R3 J L I3 ^J L? i4> J 



6. Explain the outline of the process flow for movement in the preparatory work and the 
position wishing autonomous instruction in the method 1 on the assumption that the 
explanation item of beyond outline of process flow of movement controls by method 1 1 . 



- 5. First, it is as follows if supplemented about a preparatory work. 

(1) preceding working starting -- the above 3 - as it came out and explained, move a 
robot to the three positions CB1-CB3 one by one. 

(2) In each positions CB1-CB3, acquire the picture of mark coordinate system sigmaM 
(one piece) of mark member MK1 on the representation work 3, and acquire the data 
which expresses the sight line direction based on [ each ] dots by image processing in the 
image processing device 20. 

(3) Perform processing according to the algorithm explained by above-mentioned 5. 
within the image processing device 20, and compute and memorize the data (data of RC 
and <1C>) of camera coordinate system C. 

[0059](4) When a camera calibration is completed, move a robot to the position Pt shown 
in drawing 4 , and make the position attainment state wishing instruction over mark MK1 
appear. 

[0060](5) Acquire a picture of mark coordinate system sigmaM and acquire data which 
expresses a sight line direction based on [ each ] dots by image processing in the image 
processing device 20. 

(6) Perform processing according to an algorithm explained by above-mentioned 1 .2.4. 

within the image processing device 20, and compute and memorize position attainment 

state representation data wishing instruction (data of Mg). 

[0061] Autonomous movement in a position wishing instruction is performed by a 

processing cycle containing many following steps, as shown in a flow chart of drawing 8 . 

[Kl] <d0>, <dl>, <d-l>, <d2>, and <d-2> are calculated by image processing, and M0 is 

calculated. 

[K2] M0 calculated from <d0> calculated at data and Step Kl of Mg memorized, <dl>, 
<d-l>, <d2>, and <d-2> is compared, and both coincidence/disagreement are judged. If 
in agreement, since an attainment state to a position wishing instruction is meant, it 
progresses to Step K6. If inharmonious, since it means not having reached a position 
wishing instruction, it returns to Step Kl via Step K3 - K5. Various algorithms can be 
used for evaluation of the degree of coincidence of Mg and M0. for example, what is 
necessary is to compute the deciding index delta with a following formula (46), and just 
to suppose approach un-completing, if it is delta<epsilon (epsilon - enough - smallness - 

- a positive value) and is the completion of approach, and delta>=epsilon 
[0062] 

[Equation 9] 




♦ • • (4 6) 



[K3] The four-line procession TO of four rows showing the position and posture of the 
hand (starting point of the coordinate system set up on the flange) of a robot is calculated 
and memorized. 

[K4] Using the algorithm which explained [ above-mentioned ] with the data (data of C 
and Mg) acquired by the preparatory work, the right-hand side of (2) types is calculated 



and it asks for the perpendicular slip target position Tg. 

[0063] [K5] The movement command for going to the perpendicular slip target position 
Tg is created, a servo is passed, and a robot is moved. If the non-error of a system is 
perfect, it should result in the position attainment state wishing instruction in the above 
step Kl - one cycle of K5, but the position attainment state wishing instruction is actually 
attained in successive approximations for the error of a visual sensor, the error of 
calculation, etc. 

[0064] [K6] A robot is stopped and processing is ended. 
[Method 2] 

[1] In the embodiment by the outline method 2, the light spot formed of a laser beam is 
used as a visual target means. First, drawing 9 - drawing 14 are added to a reference 
figure, and the outline of a preparatory work and teaching work is explained. Drawing 9 
is a figure for explaining inner- 1—3 of many stages of a preparatory work, and drawing 
10 is a figure for explaining the remaining stages PR4-PR6. A figure for drawing 1 1 to 
explain inner TH1-TH3 of many stages of teaching work and drawing 12 are the figures 
for explaining stage TH4 and PR5. 

[0065] [2] Preparatory work (acquisition of position attainment state representation data 
wishing instruction, and calibration of a camera) 

(PR1) a hand of a robot first represented with the flange position 2a ~ the camera 21 - in 
addition, the laser head 22 (irradiation head of the laser generator 60 shown in drawing 2 ) 
and the reference tool 25 are attached. The laser head 22 is attached via the suitable 
adjustment mechanism 23, and can adjust now the floodlighting direction of the laser 
beam 24. The mark 26 which expresses TCP (2b) with a portion shown by the arrow 27 
near the tip of the reference tool 25 is formed. However, it needs to be cautious of this 
mark 26 being for viewing. Where the laser head 22 is attached, generally the laser beam 
24 does not enter into the mark 26 on the reference tool 25, as illustrated. 
[0066] (PR2) Direction of the laser head 22 is adjusted and it is made for the laser beam 
24 to enter into the mark 26 on the reference tool 25. Thereby, light spot is formed in a 
position of the mark 26, i.e., position 2b of TCP. The laser head 22 is henceforth fixed in 
this state. 

[0067](PR3) Photography with the camera 21 is made to perform via the robot control 
device 30 in this state, and a picture for creating position attainment state representation 
data wishing instruction is acquired. Image processing of the picture of acquired light 
spot is carried out within the image processing device 20, and data showing the direction 
of light spot (it is in TCP position 2b.) seen from a camera coordinate system (starting 
point Oc) of the camera 21 of a vector <et> is created and memorized. This data is behind 
used as position attainment state representation data wishing instruction. 
[0068](PR4) If data of <et> is acquired, the reference tool 25 will be removed. Since the 
laser head 22 is fixed, the laser beam 24 will pass along TCP (2b) in the air as illustrated. 
[0069] (PR5) Data of a vector <eh> is created and memorized as a vector showing the 
laser beam's 24 seen from the starting point Oc of camera coordinate system existence 
direction. <eh> is a vector by which the following two conditions are fulfilled. 
Conditions 1 ; it is in a flat surface which a vector <et> and the laser beam 24 stretch. 
It is a vector of a direction near a direction which looked at the laser head 22 from the 
camera 21 among <eh(s)> which fill condition 2;<eh> ** <et>. <eh> can be calculated 
by a method shown, for example in drawing 13 . 



[0070]That is, a robot is moved by jogging above the suitable flat surface (here, the 
representation work surface 3 a is used.), and the spot F of the laser beam 24 is formed. 
And by approach to the field 3 a of a robot from a state which has TCP position 2b above 
the field 3 a as shown in (1), as shown in (2), TCP position 2b makes the state of the field 
3a of coming caudad. If it puts in another way, it will be made for a position of the spot F 
to come to an opposite hand from a state of (1) across the look 21a of the camera 21 and 
the intersection N of the field 3 a which look at a TCP position (2b). 
[0071]In this state, photography with the camera 21 is performed and a picture of the spot 
F is acquired. And image processing is performed within the image processing device 20, 
and it asks for data showing the direction of the spot F seen from a camera coordinate 
system of a vector <es>. A vector <eh> is computed with a following formula (47) using 
this <es>. 
[0072] 

[Equation 10] 

<et>x <es> 

<eh>= — • • , (47) 

B<et>X <es> || 

(PR6) The calibration of the camera 21 is performed. This is for asking for the relation of 
the flange coordinate system (2a is the starting point) and camera coordinate system 
which are set up on a robot hand's flange face. Various things are available as explanation 
of the method 1 described the technique of the KIKYA rib ration of a camera (the 
technique by a mark member may be used). 

[0073][3] teaching work (TH1) by the method 2 - move a robot first above the 
representation work surface 3 a where the first position wishing instruction exists by 
jogging, and form the spot F of the laser beam 24 on the representation work surface 3a. 
Marking of the position A wishing instruction shall be beforehand carried out by the 
suitable teaching point mark A which can be recognized with a visual sensor. The flange 
position 2a in the state where it illustrated shall be set to Ps, and autonomous movement 
shall be started from the position Ps. Shift to autonomous movement from jogging can be 
considered as an automatic change which used a visual-sensor output, for example so that 
it may mention later. 

[0074]TCP position 2b is above the field 3 a, and the laser beam 24 forms the spot F on 
the field 3 a through this point as illustrated. Generally in this stage TH1, three points are 
in another position mutually [ the teaching point mark A and the spot positions F and 
TCP (2b) ]. The look 21a which looked at the teaching point mark A from the camera 21 
does not pass along TCP (2b). 

[0075](TH2) A robot is operated by processing of autonomous movement and the look 
21a which looked at the teaching point mark A from the camera 21 passes along TCP 
(2b). As operation of a robot, rotation of a circumference of the starting point Oc of a 
camera coordinate system is rational. 

[0076](TH3) A robot is operated by processing of autonomous movement, and a 
formation position of the spot F is made to approach the teaching point mark A, keeping 
conditions by which the look 21a passes along TCP (2b) as much as possible. As 
operation of a robot, rotation of a circumference of Oc should be suitably combined with 
advance of the direction of look 21a. 

[0077] (TH4) If advance of the direction of look 21a becomes superfluous, while the look 
21a will keep conditions which pass along TCP (2b) as much as possible, in accordance 



with the direction of look 21a, it retreats suitably, and rotation of a circumference of Oc is 
combined suitably. 

[0078](TH5) Hereafter, operation (advance) of TH3 and operation of TH4 (retreat) are 
repeated, and a formation position of the spot F is completed as the teaching point mark 
A. Since it is exactly an intersection of the look 21a which this convergent point passes 
along TCP (2b), and looks at the teaching point mark A, and a laser beam which passes 
along TCP (2b) therefore, it will come to the teaching point mark A and the position in 
which three points are the same as for the spot positions F and TCP (2b) after all. This 
means namely that instruction target point slack TCP (2b) reached a position wishing 
instruction. A position of flange position 2b at this time is expressed with Pt as well as a 
case of the method 1 . 

[0079]Then, if a robot position in this state is memorized to a control means (nonvolatile 
memory in a robot control device), position instruction of the position A will be 
completed. What is necessary is just to apply the above-mentioned processes TH1-TH5 
about each teaching point mark of the 2nd henceforth, when there are two or more 
positions wishing instruction (teaching point mark) (see the point Al - A4 on the 
representation work 3 in drawing 1 , for example although a graphic display is omitted). 
[0080] [4] Explain an algorithm of processing for attaining movement in the position Pt 
(similarly it expresses in a flange position) wishing instruction from the autonomous 
movement starting position Ps (it expresses in a flange position) according to a process 
explained by the processing above at the time of position autonomous movement wishing 
instruction in the method 2 [2]. Although sign <> expresses a vector as well as a case of 
the method 1, a definition content of a vector is separate from the method 1 . Autonomous 
movement is performed based on a processing cycle according to an algorithm which 
described an outline in a flow chart of drawing 15 (LI -LI 2) and drawing 16 (L13-L19). 
Hereafter, it divides into each step and explains focusing on an algorithm. The meaning 
of many signs under explanation is as follows. 

[0081]sigma0, sigmal ; A value of 1 [ **] is taken with a flag with which all express 
advance/retreat of a robot. +1 means advance and -1 means retreat. 
1; register value [ showing the amount of units of advance/retreat of a robot ] <dm>; -- to 
have been shown in drawing 14 is asked from image data by a unit vector (length 1) 
showing a sight line direction which looked at the teaching point mark A from the 
starting point Oc of a camera coordinate system. 

<ds>; to have been shown in drawing 14 similarly is asked from image data like <dm> by 
a unit vector (length 1) showing a sight line direction which looked at spot position F 
(incidence position of the laser beam 24) from the starting point Oc of a camera 
coordinate system. 

[0082]T; procession deltar which expresses a position and a posture of a robot on a robot- 
coordinates system, deltal ; Positive constant near [ in a threshold with suitable all ] 0. 
What expresses the minimum rotation with a rotation procession which describes a 
relation of the direction of R;<dm > and <et > among 3x3 processions used as R<et >= 
<dm>. 

sgn; it defines as signum as follows. 
sgn(x)=+l (x>0) 
sgn(x)=0 (x=0) 
sgn(x)=-l (x<0) 



[LI] Flag sigmaO is initialized to 1. 1 is initialized to a suitable initial value (for example, 
3 cm). 

[L2] A register value showing data of the procession T is updated based on a current 
position of a robot. 

[L3] A photograph is taken using the camera 21 and it tries to search for a vector <dm> 
using the image processing device 20. 

[L4] Since it is possible that the teaching point mark A is not caught with a camera view 
if a vector <dm> is not searched for, processing is ended (an operator is changed into the 
state where adjust a robot position by jogging and the teaching point mark A is certainly 
caught with a camera view). When a vector <dm> is searched for, it progresses to Step 
L5. 

[0083][L5] It asks for an outer product of a vector <et> and a vector <dm>, and <dr =[ >] 
<et> x <dm>. 

[L6] As compared with threshold deltar, if not small, he follows an absolute value of a 
vector <dr> to Step L7. If small, it will progress to Step L10. 

[0084] [L7] The rotation procession R which describes a relation of the direction of <dm> 
and <et> is computed by a declared formula. A declared formula is obtained by solving 
the following simultaneous equations about R. 
R<et >= <dm> ... (48) 
R<dr>=<dr>... (49) 

[L8] A register value showing data of the procession T is updated to a thing to a 
procession of a notation containing the new submatrix R. This procession solves the 
following equation (50) about Tg, and is set to Tg ->T. 
[0085] 

[Equation 11] 



[L9] A robot is operated, processing for making it move to the position and posture which 
the procession T calculated at Step L8 expresses is performed, and it returns to Step L2. 
[L10] By step L6, if it is judged that the absolute value of a vector <dr> is smaller than 
threshold deltar, this step L10 will be performed for the first time. It tries to search for the 
unit vector <ds> which expresses with this step the sight line direction which looked at 
spot position F from the starting point Oc of the camera coordinate system from image 
data. 

[0086] [LI 1] It is thought that it cannot happen that the spot F is not settled in a camera 

view when [ this ] an absolute value of a vector <dr> is about 0 (a look which looks at 

TCP, and a look which looks at a teaching point mark are real coincidence) unless there 

are a system abnormality, a design error, etc. Therefore, in almost all cases, a vector <ds> 

is searched for and it progresses to Step L12. However, processing is ended if <ds> 

should not be calculated (a cause inquiry etc. are performed separately). 

[0087] [LI 2] <ds> and <et> are the real same directions, and it confirms whether to be the 

inner product 1 (it is cautious of deltal being a very small positive value). 

[LI 3] Flag sigmal showing advance/retreat of a robot will be set to sigmal =0, if an 

inner product of <ds> and <eh> is positive, it is sigmal =-1 and negative and it will 

become sigmal =1 and a real target zero. 




[L14] If it is sigmal =0, it will progress to Step LI 5. If it is not sigmal =0, it will 
progress to Step LI 6. 

[LI 5] sigmal is updated to sigmaO and it progresses to Step LI 8. 

[LI 6] If a product of sigmal and sigmaO is positive, it will progress to Step LI 8. If it is 

negative, it will progress to Step LI 7. 

[0088] [LI 7] A value of 1 is reduced by half while updating sigmaO to sigmal. 

[LI 8] A register value showing data of the procession T is updated to a thing to a 

declared procession. This procession solves the following equation (51) about Tg, and is 

settoTg->T. 

[0089] 

[Equation 12] 



[LI 9] A robot is operated, processing for making it move to the position and posture 
which the procession T calculated at Step LI 8 expresses is performed, and it returns to 
Step L2. 

In the processing cycle according to the algorithm explained above, if the judgment 
output of yes is obtained at Step LI 2 at one of the times, it will be understood as the 
position attainment state wishing instruction as shown in (TH5) of drawing 12 having 
been realized, a robot will be stopped, and processing will be ended. 
[0090] As mentioned above, since movement in the position wishing instruction of a robot 
will be autonomously performed so that it may become the position of a position 
attainment state wishing instruction and the posture which it was inputted beforehand if 
this invention is followed so that I may be understood from the explanation about the 
embodiment of the method 1 and the method 2, working efficiency improves. 
[0091]the method 1 and the method 2 -- also in which embodiment, it is preferred to 
automate a change in the mode of autonomous movement in which it tends toward a 
position wishing instruction autonomously, from the mode of jogging by a manual 
command input by processing inside a control means (the robot control device 30 and the 
image processing device 20). 

[0092] Drawing 17 shows an operating procedure of a teaching control panel, and outline 
of processing with a flow chart about a case where a change in the mode of autonomous 
movement of jogging from the mode is performed inside a system. It is as follows when 
processing of each step is described briefly. An output of a visual sensor is used for 
determination of timing of a mode change in this example. 

[SI] If it judges whether the jogging button 47 (any one) was pushed and instructions of 
jogging were made, and are made, will progress to Step S2 and it will not be made, Step 
SI is repeated. 

[S2] According to movement contents (a direction, an axis, etc.) specified with the 
jogging button 47, a predetermined axis is controlled and jogging movement of the robot 
10 is started. For example, if movement of the direction of J6 axis + is specified, a 
movement command which advances J6 axis in the direction of + will be created, and a 
servo will be passed. 

[S3] It judges whether the jogging button 47 was turned OFF, and if are turned OFF and 
Step S8 will not be progressed and used, it progresses to step S4. 




• • • (51) 



[S4] If existence of reception of a signal which tells that the teaching point mark A (a 
case of the method 2 and the spot F) entered in a view of the camera 13 was checked, it 
has received and it will not have progressed and received to Step S5, it returns from the 
image processing device 20 to Step S3. 

[S5] Processing of autonomous movement is started. It is as having divided into the 
method 1 and the method 2 and having mentioned above about an example of the 
concrete contents of processing. 

[S6] It judges whether the jogging button 47 was turned OFF, and if are turned OFF and 
Step S8 is not progressed and used, it progresses to Step S7. 

[S7] It judges whether it reached to a position wishing instruction, if it has reached, it will 
progress to Step S8, and if it has not reached, it will return to Step S6. It is as having also 
described a judgment method which is not reached [ the position attainment wishing 
instruction / ] in explanation of the method 1 and the method 2. 
[S8] Operation of each axis of the robot 2 is stopped. 
[0093] 

[Effect of the Invention] According to this invention, since the data of the position on the 
picture of the visual target means observed when the position wishing instruction is 
reached is inputted beforehand, using a visual target means like a navigation index, a 
robot can be turned to the position wishing instruction and can be moved autonomously. 
Therefore, the burden of the operator which the position teaching work of a robot takes is 
eased substantially. The highly convenient nature of the robot at the time of position 
teaching work can also be further raised by performing the shift to autonomous 
movement from jogging with the automatic change in a system especially. 



TECHNICAL FIELD 



[Field of the Invention] Concerning the movement control system for position instruction 
of a robot in more detail, this invention relates to the movement control system for 
moving a robot to the position wishing instruction quickly using a visual target and the 
visual sensor which recognizes this. 



PRIOR ART 



[Description of the Prior Art]Most generally as a position teaching method of a robot, the 
method by jogging (robot motion by a manual input) is used. According to this teaching 
method, an operator operates the jogging button of a teaching control panel, and moves a 
robot to the position wishing instruction, and the work of teaching the position at that 
time to a robot is done. When off-line instruction is made and coarse position instruction 
has ended, or when making correction of teaching positions, fine adjustment of the 
teaching positions by jogging is performed. 

[0003]use any -- since the work for which a robot is moved to it locating [ to wish ] is 
done by viewing with slight accuracy in relative position and posture relation of a robot 
hand (or end effector) and the point wishing [ instruction ], an operator takes skill to it. 
As for the number of points (a position and a posture) to be taught, since it is usually to 



attain to a large number, it is not new to spend great time on teaching work, either. The 
teaching work depending on viewing of an operator tended to produce variation for 
instruction accuracy, and there was a problem also in respect of reliability. 



EFFECT OF THE INVENTION 



[Effect of the Invention] In this invention, the data of the position on the picture of the 
visual target means observed when the position wishing instruction is reached is inputted 
beforehand. 

Therefore, using a visual target means like a navigation index, a robot can be turned to 
the position wishing instruction and can be moved autonomously. 
Therefore, the burden of the operator which the position teaching work of a robot takes is 
eased substantially. The highly convenient nature of the robot at the time of position 
teaching work can also be further raised by performing the shift to autonomous 
movement from jogging with the automatic change in a system especially. 



TECHNICAL PROBLEM 



[Problem(s) to be Solved by the Invention]This invention is made in order to conquer 
such a situation. That is, the purpose of this invention eases the burden which takes for an 
operator at the time of teaching work, and there is in providing the movement control 
system for position instruction of the robot which enabled improvement in the efficiency 
of teaching work, and reliability. 



MEANS 



[Means for Solving the Problem]This invention was made in view of a problem of the 
above-mentioned conventional technology, and an aforementioned problem is solved by 
enabling it to move a robot to a position wishing instruction autonomously using a visual 
sensor and a suitable visual target means. A position attainment state storage step wishing 
instruction which a control means of a system which contains a robot and a visual sensor 
in many stages of a method of this invention for position attainment state representation 
data wishing instruction expressing a position attainment state wishing instruction of a 
robot is made to memorize, An impaction efficiency stage wishing instruction of moving 
a robot to a position wishing instruction by movement controls of a robot by this control 
means, and a teaching-positions storage step which makes a control means memorize 
instruction positional data are included. 

[0006]And it is the important feature of this invention that an autonomous movement 
execution phase made to carry out to this impaction efficiency stage wishing instruction 
autonomously based on position attainment state representation data wishing instruction 
which had movement in a position wishing instruction memorized is included. 
[0007]In order to make it possible, this autonomous movement position attainment state 
representation data wishing instruction, It is acquired through seeing from a visual sensor 



expressed using a visual target means which can be recognized with a visual sensor, and 
said visual sensor recognizing a position attainment equivalent state wishing instruction 
equivalent to a position attainment state wishing instruction. 

[0008JA visual target means is used in an autonomous movement stage as a guidance 
visual target means for showing a robot to said position wishing instruction through being 
recognized by a visual sensor. 

[0009] Software processing performed within a control means for autonomous movement 
is performed so that a recognition state of a guidance visual target means by a visual 
sensor may be in agreement with a recognition state corresponding to position attainment 
state representation data wishing instruction and a robot may be derived. The visual target 
means can take a gestalt of a mark means including a mark coordinate system which can 
be recognized with a visual sensor. In that case, data showing a position and a posture of 
a mark coordinate system on a picture will be contained in position attainment state 
representation data wishing instruction memorized by a position attainment state storage 
step wishing instruction. On the other hand, this mark means is used in order to prepare a 
mark coordinate system for a position with fixed in order to provide a guidance visual 
target means for autonomous movement of a robot relative relation as a position wishing 
instruction. And a robot is derived so that a recognition state of a mark coordinate system 
by a visual sensor may be in agreement with a recognition state corresponding to position 
attainment state representation data wishing instruction by movement controls based on 
software processing performed within a control means. 

[0010] A mark member fixable on a field of a representation work made into an object of 
teaching work can be used for a mark means. On this mark member, a mark coordinate 
system is drawn by a dot pattern etc. 

[001 l]There is light spot formed on a floodlighting side of an optical beam light 
projection means supported by robot as a gestalt of another available visual target means. 
When using such light spot as a visual target means, in order to prepare a position 
attainment equivalent state wishing instruction to a visual sensor in a position attainment 
state storage step wishing instruction, it is an instruction target point (a point which it is 
going to coincide with a position wishing instruction) of a robot about light spot. Usually, 
it is tool point;TCP. A reference floodlighting side for forming in a position in agreement 
is arranged. 

[0012]The optical beam floodlighting direction is adjusted and light spot is formed in an 
instruction target point position on this reference floodlighting side. This state is 
equivalent to a teaching-positions attainment equivalent state for a visual sensor. Then, 
light spot is recognized by a visual sensor, and position attainment state representation 
data wishing instruction is acquired so that data showing a position on a picture of light 
spot may be included. 

[001 3] An autonomous movement stage is performed where the optical beam 
floodlighting direction which removed a reference floodlighting side and was seen from a 
camera in a position attainment equivalent state wishing instruction is maintained. That 
is, movement controls of a robot in an autonomous movement execution phase are 
performed, using light spot which floodlights an optical beam and is formed on a field 
where a position wishing instruction exists as a guidance visual target means. Software 
processing for movement controls of a robot is performed so that a recognition state of 
light spot by a visual sensor may be in agreement with a recognition state corresponding 



to position attainment state representation data wishing instruction and a robot may be 
derived. 

[0014]In advance of autonomous movement to a position wishing instruction, it may be 
made to jog in order to perform preliminary approach to a position wishing instruction. 
As for shift to an autonomous movement stage from a jogging stage, in that case, it is 
preferred to be automatically carried out by the processing within a control means based 
on an output of a visual sensor. 

[001 5] Software processing in an autonomous movement execution phase to a position 
wishing instruction, Processing which compares with position attainment state 
representation data wishing instruction data showing a position on a picture of a guidance 
visual target means recognized by a visual sensor at the time concerned, Suppose that 
processing for carrying out the movement controls of each axis of a robot based on a 
comparison result and processing which judges completion / un-completing of a position 
attainment state wishing instruction are repeated sequentially until a judgment of position 
attainment state completion wishing instruction is made. 

[0016]Data of a position on a picture of a visual target means observed in this invention 
when a position wishing instruction is reached is inputted beforehand, and at the time of 
autonomous movement to a position wishing instruction. A visual target means is used 
like a navigation index of robot movement by observing with a visual sensor, and a robot 
is made to turn and derive to a position wishing instruction. 
[0017]By this feature, a position attainment state wishing instruction is attained 
efficiently, without applying a burden to an operator. The highly convenient nature of a 
robot at the time of position teaching work can also be further raised by performing shift 
to autonomous movement from jogging with the automatic change in a system. 
Therefore, even when a large number [ the number of positions wishing instruction ], 
there are few burdens of teaching work and they end. 
[0018] 

[Embodiment of the Invention] Drawing 1 is a figure which illustrates notionally the 
overview of the movement control system for the position instruction of a robot 
according to this invention. The numerals la express the position attainment state 
representation data input means wishing instruction which inputs the data expressing the 
position attainment state wishing instruction. This position attainment state representation 
data input means la wishing instruction, Visual target Bl Data expressing the relative 
physical relationship of the visual target Bl in the state where movement in the position 
wishing instruction of the robot [ / else ] 2 was completed, and the robot hand 2a. (it is 
hereafter called "the position attainment state representation data wishing instruction".) - 
it has the function to input into the memory 4 beforehand. 

[0019]The position wishing instruction is illustrated by the four numerals Al on the field 
3 a of the representation work 3 - A4. Bl drawn Al - near each [ of A4 ] - B4 are given by 
the position Al wishing instruction - A4 representing the position which Al - A4 express 
which are visual target means and each correspond, and a fixed relation. The example of 
a visual target means is mentioned later. In the usual gestalt, each visual target position 
Bl - B4 and each position Al wishing instruction - A4 are given so that coincidence may 
not be carried out, although it is close, but they can also coincide both depending on the 
case. 

[0020]The numerals 2a and 2b express the end point position and instruction target point 



(usually tool point; TCP) of the robot respectively. In the following explanation, the end 
point position 2a of a robot shall be represented with the starting point of the coordinate 
system set up on the flange at the tip of a final arm. 

[0021] An operator operates the robot manual operation means (robot motion instruction 
manual input means; teaching control panel etc.) 5, such as a jogging button, makes the 
shaft movement control means 6 control each axis of the robot 2, and makes movement in 
the position wishing instruction start in the time of actual work. The movement controls 
for autonomous movement in the position wishing instruction from the time of this move 
start or a certain point in time after a move start are performed. The position attainment 
state representation data wishing instruction into which the movement controls for this 
autonomous movement were inputted by the position attainment state representation data 
input means la wishing instruction, It is carried out based on the relative-position- 
recognition data showing the relative physical relationship between the robot hand and 
visual target means (camera fixed to this) at each time. 

[0022]The latter data is given once [ at least ] in the process of movement controls by the 
relative-position-recognition means lb between robot hand visual targets, in the position 
attainment state wishing instruction - naturally - TCP2b ~ the position wishing 
instruction - one is in agreement with Al (or A2 - A4 one) (it illustrates by numerals 
2b'). 

[0023]The position attainment state representation data input means la wishing 
instruction and the relative-position-recognition means lb are materialized in the form 
where a visual sensor is used so that it may mention later. The visual target means Bl - 
B4 are materialized in the form of the light spot formed of the mark coordinate system or 
laser beam which can be recognized with a visual sensor. In the following explanation, 
the gestalt which materializes a visual target means in the form of a mark coordinate 
system is called "the method 1 ", and the gestalt materialized in the form of the light spot 
formed of a laser beam is called "the method 2." 

[0024] drawing 2 showed the outline of the hardware constitutions of the system used by 
this embodiment with the important section block diagram centering on a robot control 
device - it is. The robot control device to which the whole was directed with the 
numerals 30 was equipped with the processor board 31, and this processor board 31 is 
provided with the central processing unit (henceforth CPU) 31a, ROM31b, and RAM31c 
which consist of microprocessors. 

[0025]CPU31a controls the whole robot control device according to the system program 
stored in ROM31b. A program, a related preset value, etc. which defined the processing 
by the side of a robot required to perform autonomous movement to the position wishing 
instruction else [, such as a created operation program and various preset values, ] 
according to the method 1 or the method 2 are stored in RAM31c. A part of RAM31c is 
used for temporary data storage for the computation etc. which CPU3 la performs. The 
hard disk drive etc. which were suitably prepared as an external device are used for 
preservation of program data or a preset value. 

[0026]The processor board 3 1 is combined with the bus 37, and instructions and transfer 
of data are performed among other portions in the robot control device 30 via this bus 
combination. First, the digital-servo control circuit 32 is connected to the processor board 
31, and the servo motors 51-56 are driven via the servo amplifier 33 in response to the 
instructions from CPU31a. The servo motors 51-56 which operate each axis are built in 



the mechanism part of each axis of the robot 2. 

[0027]The serial port 34 having a communication interface is connected to the teaching 
control panel 40, the image processing device 20, and the laser generator 60 with a liquid 
crystal display section while being combined with the bus 37. However, the laser 
generator 60 is used by the method 2, and is unnecessary in the method 1 . 
[0028]The teaching control panel 40 has the size and weight which are the grades whose 
carrying is possible with an operator, and the jogging button etc. which are used as a 
robot manual operation means are provided on the panel. In addition, the input/output 
device 35 for digital signals (digital I/O) and the input/output device 36 for analog signals 
(analog I/O) are combined with the bus 37. When signal transfer with an end effector is 
required, the control section of an end effector is connected to digital 1/035 or analog 
1/036. In the example mentioned later, since the arc welding robot's application is 
considered, the electric power unit of an arc welding torch is connected to digital 1/035. 
[0029]The image processing device 20 is the usual thing which made CPU carry out bus 
combination of program memory, a frame memory, an image processing processor, data 
memory, the camera interface, etc. The camera 21 is connected to the image processing 
device 20 via the camera interface. This camera is used for the photography for acquiring 
the picture of a visual target means in the mode mentioned later. The program data for 
image analyses needed by the method 1 or the method 2 is stored in program memory. 
[003O]Drawing_3_is a figure showing the outline composition of the panel surface of the 
teaching control panel 40. The display screen 41 is a liquid crystal display, and the 
detailed data of a movement command program, etc. change it, and it is displayed. The 
function key 42 is a key which selects the menu displayed on the lower end part of the 
display screen 41. The teaching control panel effective switch 43 is a switch which 
switches whether operation of the teaching control panel 40 is effective or invalid. 
[003 l]The emergency stop button 44 is a button to which the emergency stop of the 
operation of the robot 2 is carried out. The cursor key 45 is a key which moves the cursor 
displayed on the display screen 41 . A numerical keypad and other keys are provided in 
the ten key part 46, and input of a numerical value and a character, deletion, etc. can be 
performed to it. 

[0032] Although a series of jogging buttons 47 (J1-J6) are buttons which specify 
advancing side by side / hand of cut, and the direction of +-, and input a movement 
command in the normal mode which jogs a conventional system, According to this 
embodiment (autonomous movement mode), it is used as an autonomous movement 
command input means to the position wishing instruction so that it may mention later. 
The details of an embodiment are explained below about the method 1 and the method 2 
on the assumption that the above matter. 
[0033][Method 1] 

[1] The schematic diagram 4 is a figure explaining operation of the autonomous 
movement in the embodiment by the method 1. On the field 3 a of the representation work 
directed with the numerals 3 as well as drawing 1 , a number (here four pieces) 
corresponding to the number of the teaching points Al - A4 to wish of mark members 
MK1-MK4 are stuck. On each mark member, the same mark coordinate system is drawn 
by the dot pattern so that it may mention later. Mark member MK1 is stuck by the 
position and the posture in which it corresponds to the position and posture of the point 
Al wishing [ instruction ] correctly. Similarly, the mark members MK2-MK4 are stuck 



by the position and the posture in which it corresponds to the point A2 wishing [ 
instruction ] - the position and posture of A4 correctly respectively. 
[0034]The robot which carried out representation only of the hand part circumference 
carries the welding torch 2c and the camera 21 in the hand part, and TCP2b is set up at 
the tip of welding torch 2c. By this embodiment, a robot is left from the move starting 
position Ps, autonomous movement of the TCP2b set up at the tip of the welding torch 2c 
is carried out to the point Al wishing [ instruction ] - A4 one by one, and the case where 
position instruction is performed in each position wishing instruction is considered. 
Generally the starting position Ps of autonomous movement is arbitrary, if mark MK1 of 
the beginning is a position included in the view of the camera 21 . 
[0035]Drawjng_5is a figure explaining the composition of the mark member used by the 
embodiment by the method 1 . Reference of the figure constitutes mark coordinate system 
sigmaM from the five circular dots DO arranged in the shape of a lattice by the interval a 
on the mark member MK, Dl, D-l, D2, and D-2. Dot space a is positive constant value. 
Therefore, the center position of each dot is set to DO (0, 0, 0), Dl (a, 0, 0), D-l (- a, 0, 
0), D2 (a, a, 0), and D-2 (- a, a, 0). As long as it can express a three-dimensional 
rectangular coordinate system, a mark coordinate system may consist of other patterns. 
[0036]Hole MH is provided in the regular position where the mark member MK is 
suitable. This hole MH points to the position wishing instruction, and when sticking on 
the field 3a of the representation work 3, a sticking position is chosen so that that 
representative point (for example, center) may be in agreement with the position wishing 
instruction. A pasting posture is chosen with reference to direction of mark coordinate 
system sigmaM. That is, if pasting postures differ, the postures taught later also differ. 
For example, since the pasting posture of MK1 and MK4 in drawing 4 is different 90 
degrees, it becomes what differed also in the posture taught later 90 degrees. 
[0037] [2] Preparation (the calibration of a camera, and acquisition of the position 
attainment state representation data wishing instruction) 

1 . Perform the calibration of the camera with the application of suitable calibration 
method in advance of working starting. Although various techniques are known by the 
calibration of the camera, the mark member shown here can also be used. The outline is 
described on the convenience of explanation, and in the back. 

[0038J2. Perform acquisition of the position attainment state representation data wishing 
instruction. First, the mark member MK is fixed to a suitable position. Of course, one of 
the mark members stuck on the representation work 3 can also be used as it is. First, by 
jog operation of the normal mode (conventional system), a robot is moved, tool point 2b 
is coincided with representative point MA of hole MH of the mark member MK, and it is 
made to take the posture wishing instruction. It means that the position attainment state 
wishing instruction was realized about the mark member MK by this. 
[003 9] 3. If the position attainment state wishing instruction is realized about the mark 
member MK, photography with the camera 21 will be made to perform via the robot 
control device 30, and the picture for creating the position attainment state representation 
data wishing instruction will be acquired. 

4. Image processing of the acquired picture is carried out within the image processing 
device 20, and the data showing the relative positional attitude of mark coordinate system 
sigmaM to the camera coordinate system of the camera 21 is created (it mentions later for 
details). 



[0040] [3] Start movement in the position wishing autonomous instruction from the 
approach starting position Ps (it expresses in a flange position) shown in processing 
drawing 4 at the time of the position autonomous movement wishing instruction in the 
method 1, and explain the algorithm of the processing for attaining movement in the 
position Pt (it expresses in a flange position) wishing instruction. In the following 
explanation, since a vector is expressed, sign <> is used. 

1 . Skeleton drawing 7 of an algorithm describes the skeleton of an algorithm required for 
movement in the autonomous position wishing instruction. The current position (position 
at each [ under movement in the position wishing instruction ] time) TO of a flange, and 
the present relative position (relative position at each [ in the moving process to the 
position wishing instruction ] time) MO of a camera coordinate system and a mark 
coordinate system, As [ describe / the geometric relation with the flange position Tg and 
relative-position (target relative position) Mg of a camera coordinate system and a mark 
coordinate system in the position attainment state wishing instruction in the position 
attainment state wishing instruction / into drawing 4 ] 

[0041] And Mg is equivalent to the position attainment state representation data wishing 
instruction acquired by the above-mentioned preparatory work. C is taught by a camera 
calibration. The fundamental equation which specifies the relation between these [ TO ], 
M0, Tg, and Mg is as follows. 

TO CM0 =Tg CMg ... (1), therefore the following formula (2) which solved this about Tg 
turn into a basic equation showing the moving-target position on the rectangular 
coordinate system of a robot. 
Tg=T0 CM0 Mg'C 1 ... (2) 

If the position instruction which makes this Tg a final moving-target point is given to a 
servo, a robot can be turned to the position wishing instruction and can be moved 
autonomously. Therefore, defining the algorithm of the autonomous movement to the 
position wishing instruction results in the problem which asks for the right-hand side of 
(2) types concretely. 

[0042] (2) Among the right-hand side of a formula, TO expresses the present position data 
of the robot, and is data of the character obtained within a robot control device at any 
time. The data of C is separately gained by a suitable camera calibration (the example of 
a calibration is mentioned later). Then, it explains from how to calculate M0 and Mg. 
[0043]2. Each of equations M0 of M0 and Mg and Mg expresses the position and the 
posture of mark coordinate system sigmaM seen from the camera coordinate system (at 
the time of the completion of the present and approach). 

[0044] Since it is positive constant value, the arrangement interval a of the circular dots 
DO which constitute mark coordinate system sigmaM (refer to drawing 5 ) used by this 
embodiment, Dl, D-l, D2, and D-2 is a> 0... It is (3). M0 and Mg are the homogeneous 
transformation processions of 4x4 expressing the relation between the position and 
posture between three-dimensional rectangular coordinate systems, and can be placed 
like a following formula (4). 
[0045] 
[Equation 1] 



Here, 3x3 processions for which RM expresses rotation, and 1M are 3x1 processions 
(vector) showing a position. It considers asking for the equation which RM and 1M fill 
hereafter as an equation which MO and Mg fill. First, the following formula (5) and (6) 
defines a vector <el> and <e2>. 
[0046] 
[Equation 2] 

<e1 >= 111 ... (5) 



.-[!] 



As for the information acquired from a visual sensor about mark coordinate system 
sigmaM, the center of each circular dots DO, Dl, D-l, D2, and D-2 aims (sight line 
direction) to be in sight from the starting point of a camera coordinate system. These 
directions can be expressed with five unit vectors <d0>, <dl>, <d-l>, <d2>, and <d-2> 
as shown in drawing 6 . 

[0047]deltaij defined by the following formula (7) is introduced about the inner product 
of these vectors. 

deltaij=<di>- <dj> (i, j=-2, -1, 0, 1, 2) ... (7) - here, since all the length of <d0>, <dl>, 

<d-l>, <d2>, and <d-2> is 1, the absolute value of deltaij does not exceed 1. 

| deltaij |<=1 (a necessary and sufficient condition for equal mark formation is i=j) — (8) 

Although distance from the camera coordinate system starting point to each dot center is 

unknown, this is placed with ti (i=-2, -1, 0, 1, 2). Naturally these take a positive value. 

[0048] 

ti >0 (i=-2, -1, 0, 1, 2) ... If (9) is carried out, following equation (10) - (12) will be 
obtained. 

tO <d0> =1M ... (10) tl <dl> =lM+aRM <el> ... (11a) 
t-Kd-l> =1M -aRM <el> ... (1 lb) 
t2 <d2> =tl <dl>+aRM <e2> ... (12a) 
t-2<d-2> =t-Kd-l>+aRM <e2> ... (12b) 

Therefore, if an equation of these is solved about RM and 1M and it goes, calculating ti, 
M0 and Mg can be defined. A concrete solution will be explained by a paragraph of how 
to calculate 4.M0 and Mg, and C showing a position and a posture of a camera coordinate 
system is described briefly. 

[0049] 3. The equation C of C is a procession expressing a position and a posture of a 
camera coordinate system seen from a flange, and the data is obtained by calibration of 
the camera 21 . Although various techniques are publicly known about a calibration and 
they can be used, even if it uses mark coordinate system sigmaM, a calibration of the 
camera 21 can also be performed. Here, only the main point is described briefly. 
[0050]In three different points as shown with the numerals CB1-CB3 in drawing 4 , 
photography with the camera 21 is performed respectively, image processing is 
performed and the data Pi (i= 1, 2, 3) about a relative position between the starting points 
of mark coordinate system sigmaM and a camera coordinate system is obtained. If data 
Ti (i= 1 , 2, 3) of a robot position at the time of they and each photography is combined, 



the following 3 sets of equations will be obtained. The point which calculates Pi (i= 1, 2, 
3) is the same as that of MO or Mg. 
Tl CP1 =T2 CP2 =T3 CP3 ... (13) 

Under conditions from which three postures of a robot of CB1-CB3 differ mutually, it is 
possible to solve this equation about C. This condition is set to (14). 
Conditions: Tl and T2 differ from each posture ingredient of T3 mutually. ... Data of C 
(14) Called for is memorized by a robot control device or image processing device. 
About a solution of an equation, it explains supplementarily depending on how to ask for 



[005 1]4. In MO and Mg asking, and solving equation (10) - (12) here and there, and 
calculating RM and 1M, consider calculating ti (i=-2, -1,0, 1,2) first. Therefore, a 
following formula (15) is replaced. 
[0052] 
[Equation 3] 



And equation (10) - (11) is transformed like following formula (16) - (18) using this. 
<rM> =t0 RM" 1 <d0> ... <rM>+a<el >=tl RM" 1 <dl> ... <rM>-a<el >=t-lRM" ] <d-l> ... 
(18) - here, (17) (16) If RM" 1 is cautious of it being a procession showing rotation, a 
following formula (19) will be materialized. 

(RM -1 <di>) -(RM" 1 <dj>) =<di> - <dj> (i, j=-2, -1, 0, 1, 2) ... If this (19) (19) type, and 
(5) - (8) type and (16) - (18) type are used, following expression-of-relations (20) - (23) 
will be obtained. 
[0053] 

||<rM> || 2 =t0 2 ... (20) ||<rM> || 2 +2 arMx+a 2 =tl 2 ... (21a) 
||||[ <rM> ] 2 -2 arMx+a 2 =t-l 2 ... (21b) 
||<rM> || 2 +arMx=t OtldeltaO and 1 — (22a) 
||<rM> || 2 -arMx=t0 t-ldeltaO and -1 — (22b) 
delta 0, 1 2 <1, delta 0, and -1 2 <1 — (23) 

If it arranges carefully from these formulas for ||<rM> || 2 +arMx, adeltaO, 1 and ||<rM> || 2 - 
arMx, and adeltaO and -1 to become a same sign from conditions (3), (9), a formula (22a), 
and (22b) respectively, following formula (24a) - (25b) will be obtained. 
[0054] 

[Equation 4] 
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If a formula (24a) and (24b) change further, the following formula (26) and (27) will be 
obtained. 

||<rM> || 2 =(a 2 beta+ 2 )/(l+beta- 2 ) ... (26) rMx=(abeta+beta-)/(l+beta- 2 ) ... (27) however 
beta+, and beta- shall be defined by a following formula (28a) and (28b). 
beta+ = (alpha++alpha-) / 2 ... (28a) 
beta- = (alpha+-alpha-) / 2 ... (28b) 

And if the formula (26) and (27) is substituted for the formula (20), (21a), and (21 b) and 
the conditions (3) and (9) are used, tO, tl, and t-1 can be found with the following 
formula (29), (30a), and (30b). 
[0055] 

[Equation 5] 
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t2, t-2, <1M>, and RM are further calculated as follows using these. 

[0056] 

[Equation 6] 
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5. Solve the above-mentioned equation (13) under the conditions (14) of the how [ to ask 
for C ] above-mentioned. Modification of an equation (13) will obtain the following (31) 
and (32). 

C'Tl - ! T3 C=P1 P3 - 1 ... C , T2 " J T3 C=P2 P3 ... (32) - replacement which followed 
following formula (33) - (37) here is performed. (31) 
[0057] 
[Equation 7] 
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Then, the equation (31) and (32) is decomposed into following equation (38) - (41). 
RC'Rl RC =R2 ... (38) RC _1 R3 RC =R4 ... (39) <(I-R1) 1C> =<11>-RC <12 > ... (40) <(I- 
R3) 1C> =<13>-RC <14> ... (41) 

Therefore, it becomes a target to calculate RC and <1C>. Here, a form of an equation (38) 

and (39) shows that both similar transformation processions expressing the similarity Rl 

during a rotation procession - R2, R3 - R4, and those similarity relations are RC. Then, 

the following formula (42) and (43) is obtained considering a vector showing a hand of 

cut of Ri (i= 1, 2, 3, 4) as <vi> (i= 1, 2, 3, 4). Here, <vi> (i= 1, 2, 3, 4) can be uniquely 

defined from conditions (14), a definitional equation (34), and (36). 

<vl> =RC <v2 > ... (42) <v3> =RC <v4> ... (43) 

RC can be calculated if a relation of a following formula (44) is used. 

(<vl>, <v3>, and <vl> x <v3>) =RC (<v2>, <v4>, and <v2> x <v4>) ... About (44) 

equations (40) and (41), a form of a following formula (45) is used at a bundle, and if a 

least-squares method is applied, <1C> will be calculated. 

[0058] 

[Equation 8] 
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6. Explain the outline of the process flow for movement in the preparatory work and the 
position wishing autonomous instruction in the method 1 on the assumption that the 
explanation item of beyond outline of process flow of movement controls by method 1 1 . 
- 5. First, it is as follows if supplemented about a preparatory work. 
(1) preceding working starting - the above 3 - as it came out and explained, move a 
robot to the three positions CB1-CB3 one by one. 



(2) In each positions CB1-CB3, acquire the picture of mark coordinate system sigmaM 
(one piece) of mark member MK1 on the representation work 3, and acquire the data 
which expresses the sight line direction based on [ each ] dots by image processing in the 
image processing device 20. 

(3) Perform processing according to the algorithm explained by above-mentioned 5. 
within the image processing device 20, and compute and memorize the data (data of RC 
and <1C>) of camera coordinate system C. 

[0059](4) When a camera calibration is completed, move a robot to the position Pt shown 
in drawing 4 , and make the position attainment state wishing instruction over mark MK1 
appear. 

[0060](5) Acquire a picture of mark coordinate system sigmaM and acquire data which 
expresses a sight line direction based on [ each ] dots by image processing in the image 
processing device 20. 

(6) Perform processing according to an algorithm explained by above-mentioned 1.2.4. 

within the image processing device 20, and compute and memorize position attainment 

state representation data wishing instruction (data of Mg). 

[0061] Autonomous movement in a position wishing instruction is performed by a 

processing cycle containing many following steps, as shown in a flow chart of drawing 8 . 

[Kl] <d0>, <dl>, <d-l>, <d2>, and <d-2> are calculated by image processing, and M0 is 

calculated. 

[K2] M0 calculated from <d0> calculated at data and Step Kl of Mg memorized, <dl>, 
<d-l>, <d2>, and <d-2> is compared, and both coincidence/disagreement are judged. If 
in agreement, since an attainment state to a position wishing instruction is meant, it 
progresses to Step K6. If inharmonious, since it means not having reached a position 
wishing instruction, it returns to Step Kl via Step K3 - K5. Various algorithms can be 
used for evaluation of the degree of coincidence of Mg and M0. for example, what is 
necessary is to compute the deciding index delta with a following formula (46), and just 
to suppose approach un-completing, if it is delta<epsilon (epsilon ~ enough ~ smallness - 
- a positive value) and is the completion of approach, and delta>=epsilon 
[0062] 
[Equation 9] 
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[K3] The four-line procession TO of four rows showing the position and posture of the 
hand (starting point of the coordinate system set up on the flange) of a robot is calculated 
and memorized. 

[K4] Using the algorithm which explained [ above-mentioned ] with the data (data of C 
and Mg) acquired by the preparatory work, the right-hand side of (2) types is calculated 
and it asks for the perpendicular slip target position Tg. 

[0063 ][K5] The movement command for going to the perpendicular slip target position 
Tg is created, a servo is passed, and a robot is moved. If the non-error of a system is 



perfect, it should result in the position attainment state wishing instruction in the above 
step Kl - one cycle of K5, but the position attainment state wishing instruction is actually 
attained in successive approximations for the error of a visual sensor, the error of 
calculation, etc. 

[0064] [K6] A robot is stopped and processing is ended. 
[Method 2] 

[1] In the embodiment by the outline method 2, the light spot formed of a laser beam is 
used as a visual target means. First, drawing 9 - drawing 14 are added to a reference 
figure, and the outline of a preparatory work and teaching work is explained. Drawing 9 
is a figure for explaining inner- 1-3 of many stages of a preparatory work, and drawing 
10_is a figure for explaining the remaining stages PR4-PR6. A figure for drawing 1 1 to 
explain inner TH1-TH3 of many stages of teaching work and drawing 12 are the figures 
for explaining stage TH4 and PR5. 

[0065] [2] Preparatory work (acquisition of position attainment state representation data 
wishing instruction, and calibration of a camera) 

(PR1) a hand of a robot first represented with the flange position 2a - the camera 21 - in 
addition, the laser head 22 (irradiation head of the laser generator 60 shown in drawing 2 ) 
and the reference tool 25 are attached. The laser head 22 is attached via the suitable 
adjustment mechanism 23, and can adjust now the floodlighting direction of the laser 
beam 24. The mark 26 which expresses TCP (2b) with a portion shown by the arrow 27 
near the tip of the reference tool 25 is formed. However, it needs to be cautious of this 
mark 26 being for viewing. Where the laser head 22 is attached, generally the laser beam 
24 does not enter into the mark 26 on the reference tool 25, as illustrated. 
[0066] (PR2) Direction of the laser head 22 is adjusted and it is made for the laser beam 
24 to enter into the mark 26 on the reference tool 25. Thereby, light spot is formed in a 
position of the mark 26, i.e., position 2b of TCP. The laser head 22 is henceforth fixed in 
this state. 

[0067](PR3) Photography with the camera 21 is made to perform via the robot control 
device 30 in this state, and a picture for creating position attainment state representation 
data wishing instruction is acquired. Image processing of the picture of acquired light 
spot is carried out within the image processing device 20, and data showing the direction 
of light spot (it is in TCP position 2b.) seen from a camera coordinate system (starting 
point Oc) of the camera 21 of a vector <et> is created and memorized. This data is behind 
used as position attainment state representation data wishing instruction. 
[0068](PR4) If data of <et> is acquired, the reference tool 25 will be removed. Since the 
laser head 22 is fixed, the laser beam 24 will pass along TCP (2b) in the air as illustrated. 
[0069] (PR5) Data of a vector <eh> is created and memorized as a vector showing the 
laser beam's 24 seen from the starting point Oc of camera coordinate system existence 
direction. <eh> is a vector by which the following two conditions are fulfilled. 
Conditions 1; it is in a flat surface which a vector <et> and the laser beam 24 stretch. 
It is a vector of a direction near a direction which looked at the laser head 22 from the 
camera 21 among <eh(s)> which fill condition 2;<eh> ** <et>. <eh> can be calculated 
by a method shown, for example in drawing 13 . 

[0070]That is, a robot is moved by jogging above the suitable flat surface (here, the 
representation work surface 3 a is used.), and the spot F of the laser beam 24 is formed. 
And by approach to the field 3 a of a robot from a state which has TCP position 2b above 



the field 3 a as shown in (1), as shown in (2), TCP position 2b makes the state of the field 
3a of coming caudad. If it puts in another way, it will be made for a position of the spot F 
to come to an opposite hand from a state of (1) across the look 21a of the camera 21 and 
the intersection N of the field 3a which look at a TCP position (2b). 
[0071] In this state, photography with the camera 21 is performed and a picture of the spot 
F is acquired. And image processing is performed within the image processing device 20, 
and it asks for data showing the direction of the spot F seen from a camera coordinate 
system of a vector <es>. A vector <eh> is computed with a following formula (47) using 
this <es>. 



(PR6) The calibration of the camera 21 is performed. This is for asking for the relation of 
the flange coordinate system (2a is the starting point) and camera coordinate system 
which are set up on a robot hand's flange face. Various things are available as explanation 
of the method 1 described the technique of the KIKYA rib ration of a camera (the 
technique by a mark member may be used). 

[0073] [3] teaching work (TH1) by the method 2 - move a robot first above the 
representation work surface 3a where the first position wishing instruction exists by 
jogging, and form the spot F of the laser beam 24 on the representation work surface 3 a. 
Marking of the position A wishing instruction shall be beforehand carried out by the 
suitable teaching point mark A which can be recognized with a visual sensor. The flange 
position 2a in the state where it illustrated shall be set to Ps, and autonomous movement 
shall be started from the position Ps. Shift to autonomous movement from jogging can be 
considered as an automatic change which used a visual-sensor output, for example so that 
it may mention later. 

[0074]TCP position 2b is above the field 3 a, and the laser beam 24 forms the spot F on 
the field 3 a through this point as illustrated. Generally in this stage TH1, three points are 
in another position mutually [ the teaching point mark A and the spot positions F and 
TCP (2b) ]. The look 21a which looked at the teaching point mark A from the camera 21 
does not pass along TCP (2b). 

[0075] (TH2) A robot is operated by processing of autonomous movement and the look 
21a which looked at the teaching point mark A from the camera 21 passes along TCP 
(2b). As operation of a robot, rotation of a circumference of the starting point Oc of a 
camera coordinate system is rational. 

[0076](TH3) A robot is operated by processing of autonomous movement, and a 
formation position of the spot F is made to approach the teaching point mark A, keeping 
conditions by which the look 21a passes along TCP (2b) as much as possible. As 
operation of a robot, rotation of a circumference of Oc should be suitably combined with 
advance of the direction of look 21a. 

[0077] (TH4) If advance of the direction of look 21a becomes superfluous, while the look 
21a will keep conditions which pass along TCP (2b) as much as possible, in accordance 
with the direction of look 21a, it retreats suitably, and rotation of a circumference of Oc is 
combined suitably. 

[0078] (TH5) Hereafter, operation (advance) of TH3 and operation of TH4 (retreat) are 



[0072] 

[Equation 10] 




< eh> = 
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repeated, and a formation position of the spot F is completed as the teaching point mark 
A. Since it is exactly an intersection of the look 21a which this convergent point passes 
along TCP (2b), and looks at the teaching point mark A, and a laser beam which passes 
along TCP (2b) therefore, it will come to the teaching point mark A and the position in 
which three points are the same as for the spot positions F and TCP (2b) after all. This 
means namely that instruction target point slack TCP (2b) reached a position wishing 
instruction. A position of flange position 2b at this time is expressed with Pt as well as a 
case of the method 1 . 

[0079]Then, if a robot position in this state is memorized to a control means (nonvolatile 
memory in a robot control device), position instruction of the position A will be 
completed. What is necessary is just to apply the above-mentioned processes TH1-TH5 
about each teaching point mark of the 2nd henceforth, when there are two or more 
positions wishing instruction (teaching point mark) (see the point Al - A4 on the 
representation work 3 in drawing 1 , for example although a graphic display is omitted). 
[0080] [4] Explain an algorithm of processing for attaining movement in the position Pt 
(similarly it expresses in a flange position) wishing instruction from the autonomous 
movement starting position Ps (it expresses in a flange position) according to a process 
explained by the processing above at the time of position autonomous movement wishing 
instruction in the method 2 [2]. Although sign <> expresses a vector as well as a case of 
the method 1, a definition content of a vector is separate from the method 1. Autonomous 
movement is performed based on a processing cycle according to an algorithm which 
described an outline in a flow chart of drawing 15 (LI -LI 2) and drawing 16 (L13-L19). 
Hereafter, it divides into each step and explains focusing on an algorithm. The meaning 
of many signs under explanation is as follows. 

[0081]sigma0, sigmal ; A value of 1 [ **] is taken with a flag with which all express 
advance/retreat of a robot. +1 means advance and -1 means retreat. 
1; register value [ showing the amount of units of advance/retreat of a robot ] <dm>; ~ to 
have been shown in drawing 14 is asked from image data by a unit vector (length 1) 
showing a sight line direction which looked at the teaching point mark A from the 
starting point Oc of a camera coordinate system. 

<ds>; to have been shown in drawing 14 similarly is asked from image data like <dm> by 
a unit vector (length 1) showing a sight line direction which looked at spot position F 
(incidence position of the laser beam 24) from the starting point Oc of a camera 
coordinate system. 

[0082]T; procession deltar which expresses a position and a posture of a robot on a robot- 
coordinates system, deltal ; Positive constant near [ in a threshold with suitable all ] 0. 
What expresses the minimum rotation with a rotation procession which describes a 
relation of the direction of R;<dm > and <et > among 3x3 processions used as R<et >= 
<dm>. 

sgn; it defines as signum as follows. 
sgn(x)=+l (x>0) 
sgn(x)=0 (x=0) 
sgn(x)=-l (x<0) 

[LI] Flag sigmaO is initialized to 1. 1 is initialized to a suitable initial value (for example, 
3 cm). 

[L2] A register value showing data of the procession T is updated based on a current 



position of a robot. 

[L3] A photograph is taken using the camera 21 and it tries to search for a vector <dm> 
using the image processing device 20. 

[L4] Since it is possible that the teaching point mark A is not caught with a camera view 
if a vector <dm> is not searched for, processing is ended (an operator is changed into the 
state where adjust a robot position by jogging and the teaching point mark A is certainly 
caught with a camera view). When a vector <dm> is searched for, it progresses to Step 
L5. 

[0083][L5] It asks for an outer product of a vector <et> and a vector <dm>, and <dr =[ >] 
<et> x <dm>. 

[L6] As compared with threshold deltar, if not small, he follows an absolute value of a 
vector <dr> to Step L7. If small, it will progress to Step L10. 

[0084] [L7] The rotation procession R which describes a relation of the direction of <dm> 
and <et> is computed by a declared formula. A declared formula is obtained by solving 
the following simultaneous equations about R. 
R<et = [>]<dm>... (48) 
R<dr=[>] <dr>... (49) 

[L8] A register value showing data of the procession T is updated to a thing to a 
procession of a notation containing the new submatrix R. This procession solves the 
following equation (50) about Tg, and is set to Tg ->T. 
[0085] 

[Equation 11] 



[L9] A robot is operated, processing for making it move to the position and posture which 
the procession T calculated at Step L8 expresses is performed, and it returns to Step L2. 
[L10] By step L6, if it is judged that the absolute value of a vector <dr> is smaller than 
threshold deltar, this step L10 will be performed for the first time. It tries to search for the 
unit vector <ds> which expresses with this step the sight line direction which looked at 
spot position F from the starting point Oc of the camera coordinate system from image 
data. 

[0086] [LI 1] It is thought that it cannot happen that the spot F is not settled in a camera 

view when [ this ] an absolute value of a vector <dr> is about 0 (a look which looks at 

TCP, and a look which looks at a teaching point mark are real coincidence) unless there 

are a system abnormality, a design error, etc. Therefore, in almost all cases, a vector <ds> 

is searched for and it progresses to Step LI 2. However, processing is ended if <ds> 

should not be calculated (a cause inquiry etc. are performed separately). 

[0087] [LI 2] <ds> and <et> are the real same directions, and it confirms whether to be the 

inner product 1 (it is cautious of deltal being a very small positive value). 

[LI 3] Flag sigmal showing advance/retreat of a robot will be set to sigmal =0, if an 

inner product of <ds> and <eh> is positive, it is sigmal =-1 and negative and it will 

become sigmal =1 and a real target zero. 

[LI 4] If it is sigmal =0, it will progress to Step LI 5. If it is not sigmal =0, it will 
progress to Step LI 6. 

[LI 5] sigmal is updated to sigmaO and it progresses to Step LI 8. 



R 




[LI 6] If a product of sigmal and sigmaO is positive, it will progress to Step LI 8. If it is 
negative, it will progress to Step LI 7. 

[0088][L17] A value of 1 is reduced by half while updating sigmaO to sigmal. 

[LI 8] A register value showing data of the procession T is updated to a thing to a 

declared procession. This procession solves the following equation (51) about Tg, and is 

settoTg->T. 

[0089] 

[Equation 12] 



[LI 9] A robot is operated, processing for making it move to the position and posture 
which the procession T calculated at Step LI 8 expresses is performed, and it returns to 
Step L2. 

In the processing cycle according to the algorithm explained above, if the judgment 
output of yes is obtained at Step LI 2 at one of the times, it will be understood as the 
position attainment state wishing instruction as shown in (TH5) of drawing 12 having 
been realized, a robot will be stopped, and processing will be ended. 
[0090]As mentioned above, since movement in the position wishing instruction of a robot 
will be autonomously performed so that it may become the position of a position 
attainment state wishing instruction and the posture which it was inputted beforehand if 
this invention is followed so that I may be understood from the explanation about the 
embodiment of the method 1 and the method 2, working efficiency improves. 
[0091]the method 1 and the method 2 ~ also in which embodiment, it is preferred to 
automate a change in the mode of autonomous movement in which it tends toward a 
position wishing instruction autonomously, from the mode of jogging by a manual 
command input by processing inside a control means (the robot control device 30 and the 
image processing device 20). 

[0092] Drawing 17 shows an operating procedure of a teaching control panel, and outline 
of processing with a flow chart about a case where a change in the mode of autonomous 
movement of jogging from the mode is performed inside a system. It is as follows when 
processing of each step is described briefly. An output of a visual sensor is used for 
determination of timing of a mode change in this example. 

[SI] If it judges whether the jogging button 47 (any one) was pushed and instructions of 
jogging were made, and are made, will progress to Step S2 and it will not be made, Step 
SI is repeated. 

[S2] According to movement contents (a direction, an axis, etc.) specified with the 
jogging button 47, a predetermined axis is controlled and jogging movement of the robot 
10 is started. For example, if movement of the direction of J 6 axis + is specified, a 
movement command which advances J6 axis in the direction of + will be created, and a 
servo will be passed. 

[S3] It judges whether the jogging button 47 was turned OFF, and if are turned OFF and 
Step S8 will not be progressed and used, it progresses to step S4. 
[S4] If existence of reception of a signal which tells that the teaching point mark A (a 
case of the method 2 and the spot F) entered in a view of the camera 13 was checked, it 
has received and it will not have progressed and received to Step S5, it returns from the 
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image processing device 20 to Step S3. 

[S5] Processing of autonomous movement is started. It is as having divided into the 
method 1 and the method 2 and having mentioned above about an example of the 
concrete contents of processing. 

[S6] It judges whether the jogging button 47 was turned OFF, and if are turned OFF and 
Step S8 is not progressed and used, it progresses to Step S7. 

[S7] It judges whether it reached to a position wishing instruction, if it has reached, it will 
progress to Step S8, and if it has not reached, it will return to Step S6. It is as having also 
described a judgment method which is not reached [ the position attainment wishing 
instruction / ] in explanation of the method 1 and the method 2. 
[S8] Operation of each axis of the robot 2 is stopped. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing l] It is a figure which illustrates notionally the overview of the movement 
control system for the position instruction of a robot according to this invention. 
[Drawing 2] the important section block diagram centering on a robot control device 
showed the outline of the hardware constitutions of the system used by this embodiment - 
- it is. 

[Drawing 3] It is a figure showing the outline composition of the panel surface of the 
teaching control panel 40. 

[Drawing 4] It is a figure explaining operation of the autonomous movement in the 
embodiment by the method 1 . 

[Drawing 5] It is a figure explaining the composition of the mark member used by the 
embodiment by the method 1 . 

[Drawing 6] In the embodiment by the method 1, a camera is a figure explaining the 
vector which describes the sight line direction which looks at a mark coordinate system. 
[Drawing 7] It is a figure explaining the outline of an algorithm required for the position 
autonomous movement wishing instruction in the embodiment by the method 1 . 
[Drawing 8] It is the flow chart which described the outline of processing of the position 
autonomous movement wishing instruction in the embodiment by the method 1 . 
[Drawing 9] It is a figure for explaining inner- 1-3 of many stages of the preparatory work 
in the embodiment by the method 2. 

[Drawing 101 It is a figure for explaining inner-4-6 of many stages of the preparatory 
work in the embodiment by the method 2. 

[Drawing 11] It is a figure for explaining inner TH1-TH3 of many stages of the teaching 
work in the embodiment by the method 2. 

[Drawing 12] It is a figure for explaining inner TH4 of many stages of the teaching work 
in the embodiment by the method 2, and TH5. 

[Drawing 13] In the embodiment by the method 2, it is a figure explaining how to search 
for a vector <eh>. 

[Drawing 14] It is a figure which illustrates a vector <dm> and <ds> about the 
embodiment by the method 2. 

[Drawing 151 It is the first half (L1-L12) of a flow chart in which the outline of the 



algorithm of processing for control of the autonomous movement in the embodiment by 
the method 2 was described. 

[Drawing 16] It is the latter half (L13-L19) of the flow chart which described the outline 
of the algorithm of processing for control of the autonomous movement in the 
embodiment by the method 2. 

[Drawing 17] In the embodiment by the method 1 or the method 2, a flow chart shows the 
operating procedure of a teaching control panel, and the outline of processing about the 
case where the change in the mode of autonomous movement of jogging from the mode 
is performed inside a system. 
[Description of Notations] 

la The position attainment status input means wishing instruction 

lb The relative-position-recognition means between robot hand visual targets 

2 Robot 

2a Flange representative point (flange coordinate system starting point) 
2b Instruction target point (TCP; tool point) 

The instruction target point (TCP) which reached the position wishing 2b' instruction 

3 Representation work 

4 Memory 

5 Robot manual operation means 

6 Shaft movement control means 

20 Image processing device 

21 Camera 

22 Laser head (irradiation part of a laser generator) 

23 The floodlighting direction control function of a laser head 

24 Laser beam 

30 Robot control device 

31 Processor board 
31a Processor 
31b ROM 

31c RAM 

35 Digital I/O 

36 Analog I/O 

37 Bus 

40 Teaching control panel 

41 Display screen 

42 Function key 

44 Emergency stop button 

45 Cursor key 

46 Ten key part 

47 (J1-J6) jogging button 
51-56 Servo motor 

60 Laser generator 

A Teaching point mark (position wishing instruction) 
Al - A4 Position wishing instruction 
Bl - B4 visual target means 

CB1-3 Position which performs the calibration of a camera 



The dot of DO, Dl, D-l, D2, and D-2 mark coordinate system 
F Spot 

MK1 -MK4 Mark member 
sigmaM mark coordinate system 
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[Translation done.] 



